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ANALYSIS  OF  A  MULTIPURPOSE 
ELECTRO  OPTICAL  SIGNAL  PROCESSOR 

INTRODUCTION 

Ail  electro  optical  signal  processor  under  investigation 
at  the  Navy  Electronics  Laboratory  Center  (NELC)  is  schemat- 
ically shown  in  Fig.  1.      Although  the  device  can  perform 
more  general  functions  involving  any  form  of  matrix  multipli- 
cation, one  of  the  primary  applications  of  the  device  is  to 
perform  integral  transforms  of  electronic  signals  (e.g. 
the  discrete  Fourier  transform) .   The  primary  features  of 
the  device  are: 

1)  It  takes  advantage  of  the  parallel  processing  capa- 
bility of  an  optical  signal  to  rapidly  perform  the 
operation.   (Operating  speed  is  limited  by  the  switch- 
ing speed  of  the  imaging  device  or  the  modulation 
capabilities  of  tne  light  emitting  diode  (LED).) 

2)  The  device  is  compact.   It  could  easily  fit  into  a 
three-inch  cube. 

3)  The  device  is  reliable.   Because  the  key  elements 
of  the  system  are  solid  state,  it  will  be  a  rugged 
reliable  device.   (In  making  this  statement,  it  is 
anticipated  that  within  five  years  commercial  CCD's 
will  be  well  developed  and  will  be  able  to  fulfill 
the  present  expectations.) 


The  scope  of  this  report  is  to  investigate  each  of  the 
elements  in  the  system  and  to  analyze  them  for  possible 
sources  of  error  in  the  output.   Another  purpose  was  to  verify 
some  of  the  implicit  assumptions  in  the  system  design  and  to 
identify  possible  problem  areas  in  the  implementation  of  the 
working  system.   Since  only  a  two  month  (May-June  19  73)  study 
was  made,  the  analysis  is  not  completely  detailed  but,  rather, 
is  of  a  preliminary  nature. 


OPERATION  OF  THE  SYSTEM 

The  operation  of  the  system  is  described  for  the  case 
of  performing  a  Fourier  transform.   Other  operations  can  be 
performed  by  using  another  transparency. 

The  discrete  Fourier  transform  can  be  considered  as 
the  following  operation  sequence: 

1)  Sampling  of  the  electronic  signal. 

2)  Multiplication  by  a  matrix  whose  elements  correspond 
to  the  value  of  the  Fourier  kernel. 

3)  Summation  of  the  products. 
Mathematically  this  is  written  as 


i   N_1        +i  2Trn3 
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where 


A(n)  is  the  n,  ,   (out  of  a  total  of  N)  Fourier  coefficient 
F{  }  is  the  discrete  Fourier  transform  operation. 
x(j)  is  the  sampled  signal  (assumed  real  and  positive). 
N     is  the  total  number  of  signal  samples  and  Fourier 

coefficients . 
In  the  electro  optic  processor,  the  input  signal  can  be 


either  discrete  (and  synchronized  with  the  detector 
switching)   or  continuous.   If  it  is  continuous,  the 
discrete  nature  of  the  imaging  device  and  the  fact  that 
it  is  switched  both  spatially  and  temporally  samples  the 
signal.   Since  an  optical  signal  is  required  to  be  real 
and  positive  (since  the  detector  is  intensity  sensitive) , 
the  processor  generally  would  require  four  channels  to 
complete  the  complex  transform  of  a  positive  real  input 
signal  (as  shown  in  Fig  2.).   One  set  of  two  channels 
handles  the  real  part,  x(j)  cos   JL  -1  ,  the  other  set  of 
two  handles  the  imaginary  part  x(j)  sin  J[   -1  .   Each 
set  has  one  channel  to  handle  the  positive  values 
and  another  for  the  negative  values  of  the  product. 
Hence,  four  parallel  channels  are  required  to  obtain  the 
complete  complex  Fourier  transform  of  the  signal.   The 
result  can  be  electronically  manipulated  to  give  the 
real  and  imaginary  parts  (or  amplitude  and  phase)  from  the 
output  terminals.   It  is  noted  that  a  bias  term  can  be 
added  to  both  the  dc  input  and  the  transparency  to  eliminate 
the  requirement  for  parallel  processing  of  the  positive 
and  negative  information. 

h  4  [v^Q[i  +  -s^] 
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where  x   and  1  have  been  added  to  the  input  signal  and 
the  Fourier  kernel  respectively. 
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The  real  part  becomes 


l   N_1  9  9   • 
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where  x ( j )  =  the  average  value  of  the  sequence 
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Subtraction  of  x   +  x ( j )  would  give  the  Fourier  coefficient 

o 

for  all  but  a(0)  which  would  require  subtraction  of 
2xq  +  x(j)  . 

A  similar  expression  holds  for  the  imaginary  parts. 
By  proper  electronic  compensation  of  the  output  of  the 
channels,  only  two  parallel  processing  channels  (Fig  3) 
are  required  rather  than  four,  as  previously  described. 
However,  the  facts  that  the  average  of  a  sequence  must  be 
computed  and  that  the  compensation  of  the  dc  Fourier 
component  is  different  from  the  others,  complicates  the 
electronic  compensation  technique. 

The  operation  of  the  system  as  a  Fourier  transform  is 
as  follows.   The  detecting  element  is  an  NxN  array  of  opt- 
cal  detectors  with  a  light  integration  capability.   Each  row 
of  N  elements  will  provide  one  Fourier   coefficient.   The 
optical  transparency  is  designed  so  that  it  also  has  NxN 
blocks  that  register  with  the  NxN  elements  of  the  receiving 
array.   Each  block  has  a  transmission  factor  that  is  propor- 


tional  to  the  kernel  of  the  transform.  For  example,  the 
element  in  row  n  and  column  j  of  the  transparency  in  the 
positive  real  channel  will  have  the  transmission  factor 


„  2irn  .  _      27m 
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The  design  of  the  transparency  is  discussed  in  a  later  section 

The  serial  switching  of  the  detector  serves  to  break 
up  a  continuous  signal  into  a  sequence.   By  following  the 
charge  accumulation  as  it  is  switched  laterally,  it  is 
seen  that  the  Nth  signal  received  from  the  n"t-n  line  is 
the  summation  of  x(j)  cos   -rr—  j    over  the  total  of  N 
elements .   (The  operation  uses  the  switching  and 
integration  capabilities  of  the  detector  in  a  "step  and 
add"  fashion.)   This  is  a  scaled  version  of  the  real 
positive  part  of  the  Fourier  coefficient.   The  negative  real 
part  and  the  imaginary  parts  are  obtained  from  the  other 
three  channels  of  the  device. 

As  this  process  continues  in  time,  the  time  varying 
Fourier  coefficients  are  obtained.   In  this  mode  of  operation, 
the  processor  acts  as  a  sliding  temporal,  window  in  sampling 
the  signal  and  the  output  is  useful  for  obtaining  a  continuous 
picture  of  the  time  evolution  of  the  spectrum.   If  nonover- 
lapping  samples  of  the  signal  are  desired,  then  the  first  N-l 


signals  from  each  line  may  be  discarded  with  the  Nth 
charge  packet  representing  the  Fourier  coefficient  for 
that  sample.   The  process  then  repeats  itself  to  determine 
the  spectral  coefficients  for  the  next  signal  sample. 

The  elements  to  be  considered  for  study  are  the  LED, 
the  transparency,  and  the  CCD  imaging  device. 


THE  LIGHT  EMITTING  DIODE 


(LED) 


The  following  properties  of  LED's  become  important  in 
the  signal  processing  scheme  envisioned. 

1.  Angular  dependence  of  radiation. 

It  is  desired  to  have  a  uniform  illumination  over 
all  parts  of  the  transparency  to  avoid  weighting  the  elements 
due  to  irregularities  in  the  beam.   The  angular  spread  of  the 
uniform  portion  of  the  LED  beam  is  easily  computed  from  the 
geometry  of  Fig.  4.   The  height,  H,  of  a  typical  CCD  imaging 
device  is  .1"  (2.54  mm).   Since  /2~  H/2=d  tan  6,  a  trade-off 
between  element  spacing  and  divergence  of  the  LED  is  required, 
For  an  arbitrary  spacing  of  1"  (2.54  cm),  the  half  angle  be- 
comes 5°.   This  is  well  within  the  capabilities  of  current 
LED's  to  provide  uniform  illumination  (e.g.  the  Fairchild 
FLV  101  and  102  have  I/I  -    .99  at  ±5°.)   At  higher  prices 
other  devices  can  provide  uniform  illumination  over  very 
wide  angles  (e.g.  Spectronics  GaAs  diode  provides  uniform 
illumination  up  to  60°.) 

2 .  Linearity  of  the  LED 

It  is  desired  to  have  the  light  follow  the  signal 
exactly.   LED's  have  excellent  linearity  to  the  current 
driving  the  device.   For  some  applications  it  might  be  neces- 
sary to  have  a  linear  voltage-to-current  converter  to  drive 
the  LED. 


3 .   Dynamic  range 

The  dynamic  range  of  the  light  output  of  the  LED's 
typically  is  in  excess  of  two  orders  of  magnitude  (40  db) , 
ranging  from  10  ma  of  input  current  to  1  amp.   At  higher  cur- 
rent levels  the  light  becomes  non-linearly  dependent  on  the 
current  due  to  thermal  effects.  To  extend  the  dynamic  range  / 
lower   current  levels  should  be  investigated  to  find  at  what 
point  the  light  intensity  deviates  from  being  linear  or  that 
noise  obscures  the  signal.   It  is  noted  that  the  linearity 
and  peak  current  fall  off  with  increasing  junction  tempera- 
ture (important  when  considering  CW  operation  about  a  dc  bias 
current) .   The  upper  current  limit  on  linear  operation  is 
determined  by  thermal  dissipation  limits  of  the  device.   With- 
out elaborate  cooling  techniques,  it  would  not  be  practical 
to  extend  the  dynamic  range  of  the  device  in  the  direction  of 
higher  currents. 

4.   Bandwidth 

Present  state-of-the-art  devices  have  a  maximum  band- 
width of  10 ' s  of  MHz.   Research  efforts  are  progressing  to 
increase  this  frequency.   It  is  noted,  however,  that  first 
generation  CCD  imaging  devices  in  their  commercial  form  are 
expected  to  operate  at  10  MHz.   Hence,  present  LED's  already 
have  enough  bandwidth  capability.   At  present  and  in  the  near 
future,  it  is  expected  that  the  bandwidth  limitations  of  the 
solid  state  imaging  device  will  be  the  primary  restriction 
on  the  system  Dandwidth  rather  than  the  light  source.   Hence, 
it  is  concluded  that  LED's  present  no  limitations  on  the  design 
and  implementation  of  the  system. 


TRANSPARENCY  DESIGN 

The  transparencies  are  required  to  be  divided  up  into 
N*N  elements  that  correspond  to  each  of  the  imaging  elements. 
Each  element  would  have  a  transmission  factor  between  0  and  1. 
Two  tecnniques  are  envisioned  for  achieving  this:  a  binary 
mask  or  a  grey  scale  photographic  mask. 

1.   Binary  mask 

The  binary  mask  uses  observations  of  various  amounts 
to  control  the  amount  of  light  falling  on  the  detector  ele- 
ment.  As  shown  in  Fig.  5  two  designs  are  possible:   one  being 
a  clear  spot  in  an  opaque  element,  the  other  being  an  opaque 
spot  in  a  clear  element.   The  transmission  factor  is  simply 
determined  by  the  ratio  of  the  area  of  the  clear  portion  to 
that  of  the  entire  element. 

For  the  first  design  this  would  be  (assuming  square 
elements)  ~ 


W 
For  the  second  design 


w'2   /  v/_\ 
"  w2  "  [  w  I 
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The  advantages  of  the  binary  transparency  are  that  it 
is  easy  to  fabricate  in  prototype  models  and  is  amenable 
to  computer-controlled  fabrication.   Since  only  two  levels 
are  involved — transparent  and  opaque--there  are  no  problems 
with  film  nonlinearities . 
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The  binary  transparency  is  subject  to  errors  due  to 
misregistration  of  the  transparency.   The  amount  of  the 
error  introduced  is  dependent  on  the  imaging  element  size, 
the  design  of  the  binary  mask,  the  transmission  of  the  ele- 
ment as  well  as  the  size  and  direction  of  the  misregistra- 
tion.  The  amount  of  the  error  for  a  given  configuration  is 
easily  determined.   The  signal  from  the  detector  is  propor- 
tional to  the  intensity  of  the  incident  light  which  in  turn 
is  determined  by  the  overlapping  areas  of  the  detector  and 
the  transparent  part  of  the  transparency.   The  overlap  of 
the  areas  is  reminiscent  of  the  spatial  correlation  of  two- 
dimensional   functions  representing  the  transparency  and  the 
detector.   This  function  is  a  maximum  at  zero  offset  (corre- 
sponding to  optimum  registration)  and  falls  off  with  displace- 
ment of  the  transparency  function.   Computation  of  the  rela- 
tive decrease  in  signal  level  with  displacement  of  the  mask 
then  can  be  computed  by  a  graphical  or  analytical  analysis 
of  the  correlation  of  the  transparency  and  the  detector. 
Investigations  into  the  sensitivities  of  the  error  for  the 
different  binary  designs  could  be  done  to  minimize  the  errors 
For  example,  the  correlations  shown  in  Fig.  6  show  that  for 
the  binary  design  with  a  clear  spot  in  an  opaque  mask,  the 
output  is  more  insensitive  to  small  displacements  for  small 
transmission  factors  than  for  large  transmission  factors. 

2 .   Grey  scale  mask 

The  second  technique  of  transparency  preparation  in- 
volves exposure  of  each  element  of  the  transparency  film  to 
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enough  light  to  create  the  proper  grey  scale.   As  long  as 
the  transparency  elements  are  larger  than  the  detector  ele- 
ments, the  errors  due  to  misregistration  are  minimized  since 
the  centering  of  the  transparency  element  is  not  critical. 
The  preparation  of  the  transparency  becomes  more  difficult, 
however.   The  individual  elements  must  be  exposed  to  a  light 
source  of  correct  intensity  and  time  duration.   Nonlineari- 
ties  of  the  film  response  should  be  taken  into  account  to 
obtain  the  correct  transmission  factor.   The  demagnified 
image  of  the  light  source  would  be  required  because  of  the 
small  size  of  the  transmission  element.   (A  detector  element 
on  a  CCD  device  is  on  the  order  of  1  mil  on  a  side.)   Since 
a  large  array  of  elements  are  required,  a  computer  controlled 
process  that  governs  the  positioning  and  exposure  (including 
the  nonlinearities)  would  reduce  the  work  to  a  tolerable 
level.   Once  a  master  mask  was  produced  in  this  way,  photo- 
graphic copies  could  be  easily  and  cheaply  produced. 

The  two  techniques  then  have  offsetting  difficulties. 
The  binary  transparencies  are  fairly  easy  to  make  but  are 
somewhat  sensitive  to  registration  problems.   Micro  positioners 
would  probably  be  required  for  accurate  registration  with  the 
small  imaging  devices  that  are  envisioned.   The  positioning 
requirement  is  not  too  difficult  to  this  scale  and  can  even 
be  partially  eased  by  using  mask  element  sizes  that  are 
smaller  than  the  detector  elements.   The  grey  seals  transpar- 
encies, while  more  insensitive  to  registration,  are  much  more 
difficult  to  make.   The  requirement  for  a  grey  scale  adds  a 
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new  dimension  to  the  fabrication  process.  The  fact  that  film 
is  nonlinear  makes  this  new  problem  even  more  difficult.  In 
this  case,  the  use  of  a  computer  controlled  fabrication  of 
a  master  transparency  would  be  required  to  reduce  the  effort 
to  a  tolerable  level.  Once  the  master  is  made,  other  trans- 
parencies can  be  photographically  duplicated  at  little  cost. 
At  present  the  binary  technique  appears  to  be  the  simpler. 
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THE  IMAGING  DEVICE 

As  described,  it  is  desirable  to  have  a  small,  reliable 
solid  state  sensing  device  that  can  combine  the  detecting 
capability  with  information  processing.   One  such  device 

obtaining  technological  prominence  is  the  charge  coupled 

r2  ] 
device  (CCD) .      In  the  processor  the  CCD  serves  as  a  means 

of  collecting  the  charge  across  various  lines  of  the  image 
at  different  points  and  different  times.  Hence,  the  inte- 
gration can  be  done  in  both  time  and  space  across  the  device. 

It  should  be  noted  that  in  the  following  discussions 
typical  parameter  values  of  available  devices  such  as  the 
Fairchild  CCD  101  linear  image  sensor  are  used.   Because 
CCD's  are  under  intensive  development,  these  values  will  be 
changing.   Where  trends  and  limitations  exist,  it  will  be 
noted.   However,  the  values  used  are  representative  of  com- 
mercially available  devices  in  mid-1973.   It  is  also  noted 
that  other  devices  such  as  the  self-scanned  photodiode 
array  also  are  potential  candidates  for  this  application, 
but  that  the  fundamental  emphasis  in  the  conceived  design  was 
on  the  applications  of  CCD's. 

The  limited  dynamic  range  of  present  CCD's  for  imaging 

3 
purposes  (typically  10  )  is  further  reduced  by  the  fact  that 
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one  element  is  used  for  collecting  the  charge  across  one 
line  of  the  device.   The  saturation  level  of  each  element 
must  be  reduced  by  1/n  to  avoid  saturating  the  signal  in 
the  case  of  constant  illumination.   The  noise  currents  are 
reduced  by  only  Jl/n   because  of  their  statistical  nature. 
Hence  the  dynamic  range  --  (saturation  level/noise  level)  is 

reduced  by  «/l/n  (or  ~ 1/23)  from  the  value  used  in  imaging 

.   •      C3] 
devices . 

In  considering  the  frequency  response  of  the  signal  one 
must  consider  both  the  desired  application  and  the  design 
of  the  CCD.   Two  designs  of  area  imaging  devices  are  anticipated 
one  involves  a  lateral  movement  of  all  lines  followed  by 
a  vertical  shift  (as  shown  in  Fig.  7) .   Allowing  for  equal 
exposures  the  individual  elements  give  the  time  varying 
Fourier  coefficients.   When  the  time  varying  Fourier  coef- 
ficients are  not  of  interest,  only  the  last  column  transferred 
out  of  the  device  is  useful  since  only  it  will  be  full  summation 
over  N.   The  terms  read  out  previously  will  be  incomplete 
summations.   Assuming  that  the  array  is  500  by  500,  it  is  noted 
that  the  vertical  transfer  must  operate  at  a  frequency  of  500 
times  that  of  the  lines  since  500  elements  must  be  read  out 
of  the  vertical  shift  register  for  each  lateral  shift  of  the 
columns.   Since  the  maximum  shifting  frequency  of  the  devices 
is  anticipated  to  be  on  the  order  of  10  MHz  for  currently 
conceived  ideas  such  as  TV  cameras  and  computer  memory  devices, 
the  maximum  signal  frequency  that  can  be  processed  is  reduced 
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by  1/N  (N  is  the  number  of  vertical  elements) .   Hence  audio 
signals  (10  MHz/500  =  20  KHz)  can  produce  500  coefficients  in 
this   application  of  non  overlapping  signal  samples. 

The  second  technique  to  be  used  in  image  detectors  is 
frame  or  field  transfer.   As  shown  in  Fig.  8,  2N  lines  are 
used;  the  upper  N  are  exposed  to  the  light  (image  area)  and 
then  quickly  transferred  vertically  to  the  lower  N  lines 
(storage  area) .   While  the  upper  frame  is  being  exposed,  the 
storage  frame  is  read  out  by  shifting  the  lines  vertically  and 
reading  out  sequentially  the  bottom  line  by  horizontal  shifting. 
Again  the  fastest  clock  frequency  appears  at  the  horizontal  shift 
register,  and  the  vertical  frequency  would  be  reduced  by  — 

(N  here  is  the  number  of  horizontal  elements) .   It  is  noted 
that  in  this  scheme  the  integration  would  occur  vertically  and 
that  the  N  Fourier  coefficients  for  non-overlapping  samples 
would  be  contained  in  the  last  row  to  be  shifted  out  of  the 
storage  frame. 

Simple  modifications  in  the  fabrication  of  the  CCD  can 
be  made  to  achieve  full  use  of  the  switching  frequency  of  the 
CCD.   Applications  that  use  non-overlapping  signal  samples 
can  use  the  time  when  the  N-l  samples  of  each  line  are  being 
discarded  to  read  out  the  previously  obtained  value  of  the 
Fourier  coefficient.   In  this  way  the  signal  bandwidth  would 
be  limited  by  the  switching  frequency. 

The  switching  frequency  itself  can  be  raised  by  modifi- 
cations in  the  design  of  the  CCD.   A  new  charge  coupled  data 
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processing  device  called  the  peristaltic  charge  coupled 
device  L    has  been  recently  developed  and  is  capable 
operating  at  frequencies  of  more  than  100  MHz.   Such  a 
device,  while  now  developed  for  signal  processing  rather  than 
imaging,  will  provide  a  useful  base  when  optical  processing 
of  high  frequency  signals  becomes  an  important  application  of 
this  technique. 

In  summary  then  the  frequency  response  of  the  processor 
is: 

1.   When  using  a  conventional  commercial  CCD  with  a 

switching  speed  on  the  order  of  10  MHz  two  modes  of 

operation  are  possible. 

a.  The  instantaneously  time  varying  Fourier  coef- 
ficients can  be  obtained  byusing  every  output 
signal.   A  500  by  500  array  operating  at  10  MHz 
could  produce  500  coefficients  every  50  micro- 
seconds.  This  is  equivalent  to  using  a  "sliding 
window"  to  sample  the  signal  every  5  microseconds. 

b.  When  discrete  non-overlapping  samples  of  the 
signal  are  desired  much  of  the  information  is 
discarded  lowering  the  maximum  signal  frequency 
but  still  providing  a  large  number  of  Fourier 
coefficients  in  a  short  amount  of  time  (e.g.  500 
coefficients  every  2.5  milliseconds  for  the 
numbers  assumed  above.   Minor  modifications  to 
the  device  design  before  fabrication  can  recover 
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the  full  frequency  response  of  the  system  and  make 
it  practical  for  a  wider  variety  of  signal  processing 
applications. 
2.   Major  modications  to  the  device  design  appear  tech- 
nologically feasible  to  raise  the  frequency  capability 
of  the  device.   Research  into  high  frequency  devices 
has  been  promising. 
Other  problems  presented  by  using  devices  and  electronics 
developed  for  television  use  are  easily  circumvented. 
Removal  of  the  interlace  is  accomplished  by  modifications 
to  the  driving  electronics.   As  element  sizes  decrease 
to  improve  resolution  capabilities  more  stringent 
registration  of  the  mask  will  be  required  but  this  is 
well  within  the  limits  of  simple  micro-positioners. 
Also  as  previously  noted,  tolerances  on  mask  registration 
can  be  somewhat  related  by  making  the  masking  elements 
smaller  than  the  sensor  elements. 
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SUMMARY 

The  electro  optic  processor  discussed  is  capable  of 
processing  electronic  signals  and  producing  processed  data 
at  a  high  rate  by  combining  the  parallel  processing  capability 
of  optics  and  the  integrating  capability  of  optical  detectors. 
It  is  emphasized  that  while  this  report  has  been  framed  in 
terms  of  Fourier  transforms,  any  other  discrete  transform  or 
multiply-and-add  operation  can  be  performed.   The  light 
emitting  diode  and  mask  fabrication  present  no  technological 
problems.   The  efficient  use  of  the  frequency  capability  of 
the  CCD  calls  for  some  ingenuity  but  the  technological  means 
are  well  in  hand.   Simple  experimental  confirmations  of 
principles  of  operations  have  been  obtained     and  further 
development  of  the  technique  is  desired  and  recommended. 
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